Papers with reading comprehension (RC
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations. |
| Approach: | They propose a framework for automatic reasoning-oriented perturbation of question-answer pairs that decomposes a question into reasoning steps required to answer it and generates new question-anchor pairs. |
| Outcome: | The proposed framework generates evaluation sets for reading comprehension benchmarks and generates examples without human intervention. |
Implanting LLM’s Knowledge via Reading Comprehension Tree for Toxicity Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for toxic content detection are small language model (SLM) based and large language model(LLM) -based. |
| Approach: | They propose to implant LLM's knowledge into SLM based methods to stick to both types of models' strengths by constructing a reading comprehension tree to transfer knowledge between two models. |
| Outcome: | The proposed method can stick to both types of models' strengths . it is compared with existing methods on real-world and machine-generated datasets. |
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (P19-1)
Copied to clipboard
Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, Junji Tomita
| Challenge: | Question answering (QA) using textual sources for purposes such as reading comprehension has attracted much attention. |
| Approach: | They propose a Query Focused Extractor model for evidence extraction and multi-task learning with the QA model. |
| Outcome: | The proposed model achieves state-of-the-art evidence extraction score on hotpotQA and FEVER, which is a recognizing textual entailment task on a large textual database. |
Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Sequence encoders are crucial components in many neural architectures for learning to read and comprehend. |
| Approach: | They propose a compositional encoder that explicitly models across multiple granularities using a new dilated composition mechanism. |
| Outcome: | The proposed encoder is fast and expressive, and can model across multiple granularities. |
A Simple and Effective Model for Answering Multi-span Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for reading comprehension restrict output space to a set of single contiguous spans . multi-span questions are problematic because they require multiple inputs - a task that requires a sequence tagging problem . |
| Approach: | They propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem. |
| Outcome: | The proposed model significantly improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively. |
Connecting Attributions and QA Model Behavior on Realistic Counterfactuals (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research in interpretability of neural models has yielded numerous token attribution techniques, but it is hard to evaluate whether these explanations are faithful. |
| Approach: | They propose to use pairwise attributions to connect outputs to high-level model behavior to examine how well different attribution techniques align with this assumption on realistic counterfactuals in the case of reading comprehension (RC). |
| Outcome: | The proposed methods are better suited to RC than token-level attributions across different RC settings, and the best performance comes from a modification that was proposed to an existing pairwise attribution method. |
R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have revealed that reading comprehension (RC) systems learn to exploit annotation artifacts and other biases in current datasets. |
| Approach: | They propose a task that requires giving answers and derivations to evaluate RC systems' internal reasoning. |
| Outcome: | The proposed framework annotates 4.6k questions with 3 reference derivations and shows that it is reliable and compares with existing benchmarks. |